A. Field of Invention
This invention relates generally to client-server computing over multiple intermittent networks.
The Client-Server is an architecture that has long been in use within computing. It became popular with the advent with the Personal Computer (PC), which was used to access data on a central server. The simplest case of this is the File Server configuration where a set of files is stored on the server and accessed by the PC. Novell Netware and Sun's Network File System (NFS) are examples of technologies that were used to share files. Another popular client-server configuration is to have a thick-client application on the PC accessing a database on the server using standards such as Structured Query Language (SQL) and Open Database Connectivity (ODBC).
Tightly coupled client-server architectures indicate that the client requires access to the server to get information. If the server is not available, the client is essentially useless. Much of the client-server computing in use today is tightly coupled. The web browser is an example of a tightly coupled client-server model because it retrieves HTML from web servers. Without access to web servers, there is little that a browser can do.
Loosely coupled clients are independent of the server and access the server only when needed, such as to access or update information. Mail clients like Microsoft Outlook is a good example, where one can read and write emails while not connected to a mail server.
Most client-server computing has been deployed in a Local Area Network (LAN) or other always-on networks, eg. leased lines, dial-up. Therefore, it is not difficult to achieve a high level of reliability. There has not been a compelling need to introduce middleware to improve the reliability. Even so, applications that require transactional-level guarantees will use some kind of Transaction Processing middleware to track, audit and roll back transactions.
In addition, the assumption within a LAN is that there is only one network and that the cost of using it is essentially free. There has not been a requirement to choose among the use of multiple networks. Nor has there been a need to carefully consider the cost of using the network, eg. how many bytes are being sent or how long it is being used.
However, with the advent of wireless networks, this is all changed. Wireless networks are by their very nature intermittent since you are never guaranteed to have a clear signal. And often, there are places where there is no connection at all. Wireless carriers typically charge for usage of their network by the byte, so it becomes important how much bandwidth is being consumed. And there are multiple networks now available to computers, such as wire-line LAN, WiFi (IEEE 802.11b and successors), Wireless Wide Area Networks (GPRS, 1×RTT), Bluetooth and even a serial cable between a PDA cradle and a PC.
Another new issue raised by the use of mobile computers is the management of the devices and assets on those devices. In the current philosophy of network and system management embodied by software such as HP OpenView and CA Unicenter, there is an assumption that network elements or nodes (eg. computers, routers, switches) are always connected to the network and rarely move. It is therefore straightforward to manage the elements using Simple Network Management Protocol (SNMP) and deploy or update software on those stationary devices. However, with devices that are mobile, there is a new set of issues. These mobile devices are not always connected and if they are, they may be connected to multiple networks and therefore have multiple IP addresses. They might be shared among a group of users (eg. truck drivers who take any arbitrary handheld computer when the start their rounds). The devices must be secure but not impose a heavy price by slowing performance or sending exponentially larger packets on expensive wireless networks.
It is therefore plain that one cannot simply extend the current networking philosophy to computing on intermittent networks. In order to deploy usable and cost-effective client-server solutions that are mission-critical on intermittent networks, the goals should be transactional guarantee and manageability.
Transactional guarantee means that the system must keep functioning regardless of whether there is connectivity or not. No messages should ever be lost but they should be kept in reliable persistent storage at each step so that they can be recovered should a failure occur in the system such as a power outage. The entire system, consisting of the mobile devices and servers, must always be in a consistent state. Even when a failure occurs, the transaction should be rolled back or otherwise compensated so that there are no conflicts in any application. An example of this is when a transaction is to be committed to two applications; if one succeeds and the other fails, the one that succeeded should be rolled back so that they are both consistent. Only when both have succeeded should the transaction be committed. The system should be performant and not allow a fault to throttle the entire system, ie. cause it to stop working or go into an infinite loop and consume a lot of resources. This can happen when a message is in a queue that is fails to be committed to a target application and continues to retry constantly; this is called a “poison message” and should be immediately taken off the queue and processed differently. It should also be very resilient to faults such as badly formatted messages so that the system does not need to be restarted when responding to problems.
Manageability encompasses security, asset management, software deployment and cost control. Security covers the usual areas of authentication, authorization, encryption and non-repudiation There are many existing technologies that can meet these requirements. Mobile devices have additional requirement of remote locking when a device is reported lost or stolen. This can be done by sending a “poison pill” to kill the device and possibly destroy data or revoking the privilege to connect back to the server when it attempts to do so the next time a connection is available. Asset management refers to tracking the devices (eg. who owns it, where is it) and the management of the configurations on the device (eg. network settings, email settings). It is required that these are done set by a central system administrator and done automatically so that the user is not burdened to set up the configuration, which can be a complex and error prone process requiring much support. Another aspect of asset management is the ability to remotely run diagnostic test programs on the device. For example, the administrator might want to schedule the barcode scanner to be test every day and a report sent automatically when there is a connection so that he knows if the device needs to be brought into the office for maintenance Software deployment is an area that has received a lot of attention because of the high cost of keeping the correct versions and license of software on computers. This problem is compounded for mobile devices that you cannot physically check. Software deployment configurations must be set up by the administrator remotely, whether the device is connected or not. When a device comes on line, it must automatically know which software to update. The administrator must also be able to specify which network to be used for software deployment. For example, use the free WiFi or serial connection to update software and only use the expensive wireless WAN for sending urgent application messages. Backing up data on the device is also a requirement for devices that have substantial disk storage such as laptops. Cost control is a new requirement for wireless devices where it does matter how much bandwidth is being used. Because wireless networks are more expensive, slower and intermittent, it becomes important for an application to determine which messages should be sent on which networks. Urgent and important messages should be sent on any available network. Less urgent and important messages should wait until a cheaper network is available. Other factors might come into play, such as system or network constraints. For example, if a satellite channel is available, only the most urgent and small messages might be sent. If the time is after 5 pm or the battery is low, perhaps the pending messages should be flushed immediately on any available channel.
C. Description of Related Art
In order to enable reliable communication between applications across intermittent networks, several traditional techniques have been used. These have been adapted from LAN (Local Area Network) technologies. The major techniques are Asynchronous Messaging, Distributed Transaction Processing and Synchronization; which will be described in more detail below.
1. Asynchronous Messaging
Asynchronous Messaging has been used to integrate enterprise applications for many years. In FIG. 1(a), an Application 101 communicates to other applications using asynchronous messaging middleware. The asynchronous messaging middleware consists of a Messaging Client 102 and Messaging Server 104. The Messaging Client is a software library that is included by the Application and takes care of ensuring that a Message is sent to its intended recipient(s) via the Messaging Server. The Message consists of a Header and Body, where the Header contains envelope information such as the address of the recipient and the priority, and the Body is a collection of text that describes the content.
The Messaging Client 102 and the Messaging Server 104 communicate via a network 103. This network might be high bandwidth (eg. Local Area Network) or low bandwidth (eg. Dial-up). The network might be very reliable or intermittent. When a message is submitted by an Application 101 to be sent to another application, the Messaging Client will try to reach the Messaging Server and send the message to it. If it fails, it will store the message and automatically retry again.
There are two main ways to send messages using asynchronous message. One is point-to-point to request-response where an Application specifies the exact location of the target. For example, a message is sent to an enterprise application such as SAP or Oracle. The other method is publish-and-subscribe where a topic is specified and applications will publish to a topic or subscribe to a topic. A good example of this is the stock trading systems whether traders subscribe to stocks they are interested to track and the systems publish stock changes to a topic that corresponds to that stock ticker symbol.
There are several policies for the messaging guarantees. With the “at least once” policy, the message must be sent at least once; meaning that the message might be sent more than once and duplicates must be discarded by the application. This method requires more administrative overhead by the application but it is the very efficient. With “at most once”, the message should be sent only once but there is a small chance that it might not be sent at all (ie. the message is lost). The most rigorous policy is “exactly once” or “once and only once”, where there is a strict protocol between the Messaging Client and Messaging Server using unique message identifiers, retries and acknowledgements to ensure that the message is sent. With high reliable networks like Local Area Networks, it is likely that one of the less reliable policies is sufficient because the underlying network transport provides for the automatic resending of packets that might be involved in network collisions and are lost. In the case of unreliable or intermittent networks like wireless networks, then it is important that the more reliable policy such as “exactly once” is used.
Given the proven reliability and flexibility of asynchronous messaging, it is natural that vendors have considered extending this software paradigm to the wireless network or any other intermittent network. In FIG. 1(b), the way that most vendors have done this is simply by inserting a Gateway 108 between the Messaging Client 106 and Messaging Server 110. The Gateway is a piece of software that generally sits in the Demilitarized Zone (DMZ) of a firewall that protects a corporation's data assets. It provides protocol translation between the Messaging Client and Messaging Server. The Messaging Client 106 now talks to the Gateway 108 via the external network 107 instead of directly to the Messaging Server 110 because the Messaging Server is behind the firewall and is not directly accessible. The Gateway often provides security services between the Messaging Client and Messaging Server. The Gateway then translates and forwards the message on to the Messaging Server. It must maintain the same message guarantee policies that have been dictated by the administrator between the Messaging Client and Messaging Server. Note that the same protocol is in use so the communication between the Messaging Client and Messaging Server is very “chatty”, as indicated by the thick black arrows 107, 109. While this works well when the network is mostly connected and reliable, it is not optimum for intermittent or unreliable networks such as wireless.
2. Distributed Transaction Processing (DTP)
Asynchronous messaging removes the headache for the application developer to ensure that the message was sent to another application, but it does not guarantee a correctly completed transaction. For this to occur, distributed transaction processing (DTP) theories have been developed and standards such as the X/Open XA Interface have been defined so that transactional applications can interoperate.
DTP can be accomplished by using a Transaction Manager (TM, also known as Transaction Authority) and asynchronous messaging. Asynchronous messaging is used to guarantee the transport of messages between the Transaction Client (TC) and Transaction Manager.
FIG. 2(a) illustrates the combination of asynchronous messaging and distributed transaction processing. While it is desirable to have transaction guarantees, many corporations do not have a transaction management engine such as BEA Tuxedo or Microsoft MTS. As such, they need to write the transaction logic themselves 213, along with the business logic and connector logic. This logic includes rolling back transactions that fail for all affected applications, whether they are online or offline. It is a complex undertaking and involves a lot of code that must be written and tested thoroughly.
In addition, a traditional DTP protocol adapted from the wired network model FIG. 2(a) would require that the messaging client 202 wait for an acknowledgement from the target application; this means that it needs to hold the connection open for the message to make a round trip all the way through the transaction logic 201, 202, 203, 204, 205, 206, 213, and back through 206, 205, 204, 203, 202 and 201. This roundtrip could potentially take a long time, especially if the transaction is targeted for multiple backend applications. In this time, a timeout could have occurred between the messaging client and messaging server, which would require a new session to be established. It also consumes more bandwidth than is necessary.
Given the abovementioned deficiencies of applying the wired network model of DTP, it is desirable therefore to amend the implementation while providing the same level of transaction guarantee. This is illustrated in FIG. 2(b) where a message from an application is handed off to the gateway 210 which releases the connection right away so that the application does not have to wait for the acknowledgement from the target application. The gateway takes care of ensuring that the message is properly submitted to the transaction manager. Since the logic for the transaction manager is generic, it should not be rewritten for each application but should be a separate module 214. When a reply is generated from the server application 215, a new connection is then established to send the message to the application 207. With this model, the roundtrip is much abbreviated: 207, 208, 209, 210 and back through 209, 208 and 207. The gateway takes care of sending the message to the application: 210, 211,212, 214 and 215. Any errors, rollbacks or replies are sent to the originating application with a new connection: 215, 214, 212, 211, 210, 209, 208 and 207. This implementation is more efficient because it minimizes the connection time required and the chances for timeouts. By abstracting the transaction logic, this also dramatically reduces the code that needs to be written by an application programmer.
KonaWare implements the model in FIG. 2(b) in addition to other innovations for intermittent networks. This will be described in more detail in the disclosure section.
3. Synchronization
Synchronization is a general term that is applied to a set of technologies that compares two different datasets and makes them the same by copying the differences to each one. The use of database replication by Lotus Notes was one of the first widespread uses of synchronization. The Personal Digital Assistant (PDA) makes use of synchronization to ensure that things such as the calendar, contacts database, notes and email are up-to-date on both the PDA as well as the PC. Palm was the first company with a simple and successful synchronization mechanism. Synchronization can also be applied to any content, files or unstructured databases (eg. Avant Go). There have been many innovations in synchronization. Some use timestamps (although this requires that the date and time must be in sync at all times). Others use markers or bookmarks to indicate the last update.
Synchronization offers a very simple programming model for the application developer because they are already used to programming against a database. However, it has a major problem that occurs whenever the same row and column of a table of the client database 302 and server database 303 are changed. When the database is synchronized, there is no way of telling which update should win. This is known as a synchronization conflict. Some databases offer the option of allowing the server to always win or the client to always win, but this is too simplistic and will fail in most cases.
Synchronization works well if there is only one client application and one server application, and that both of these are controlled by a single entity as illustrated in FIG. 3(a). When a synchronization conflict occurs, that single entity is able to decide who will win. In the example of a PDA, the same person has entered an appointment in both the PDA and the PC database. When the databases are synchronized, that person will know which one is correct. The synchronization application typically raises this as an exception that needs to be manually handled.
Handling exceptions manually is bad practice for enterprise applications because there is generally no single person who can definitively resolve all the synchronization conflicts. This is clear when one considers the scenarios in FIG. 3(b) and FIG. 3(c). In FIG. 3(b), there are two client applications 305, 309 which are updating their client databases, 306, 310 respectively. Both of these could be updating the same row and column of the same table. When the databases are synchronized, there is no way of telling which application should win. This can be generalized to multiple client applications and the problems are compounded. Some implementations segment the databases such that each client database has its own copy so that it would not conflict with another client database. However, the problem of conflicts arising from the server application and client application updating the same row and column still exists.
FIG. 3(c) illustrates a typical configuration in enterprises where there are multiple client applications 311, 316 and multiple server applications 314, 315. All synchronization is done through a central database 313. Given that it is intractable to have automatic exception management of synchronization conflicts in even a simple case such as 3(a), it is impossible for this case. Certain packaged applications have been able to use database synchronization by carefully ensuring that updates are made in different rows of the table. But it is not a generalized methodology that is useful for custom applications. It is therefore not surprising that database synchronization has not been successfully deployed for many custom enterprise applications.
In order to make database synchronization work automatically, there needs to be a master server database and client databases that are subservient to it, ie. slave databases. In the configuration shown in FIG. 3(c), the server database 313 would be the master database. It is the final arbiter of updates among the databases. The server applications 314 and 315 must communicate with it transactionally. The client application 311, 316 cannot assume that an update to its client database 312, 317 is committed until it has been confirmed by the server database 313. Such updates are considered pending until there is a connection to the server database. This puts more of a burden on the client application developer and user, but it will eliminate the need to manually handle exceptions, which is much more costly in the long run.
KonaWare combines this type of database synchronization for tables that are usually static and make sense to use this technique. It is described in more detail in the disclosure section.
4. Networking
Networking has evolved over the years to standardize largely on the Internet (TCP/IP) and Web (HTTP/HTML) standards. The networking philosophy is based on separating protocols into a series of distinct layers or stacks. The OSI model is useful for understanding the networking model and is illustrated in FIG. 4(a), where:                Physical layer 407 connects devices to networks        Data link layer 406 detects and corrects errors        Network layer 405 routes the transmissions        Transport layer 404 ensures message integrity        Session layer 403 controls the start/end of a session        Presentation layer 402 translates data to the appropriate rendering format        Application layer 401 presents the information to the user TCP/IP is a set of protocols that corresponds to the OSI model as shown in FIG. 4(a) and FIG. 4(b),where:        IP 412 is a connectionless Internet Protocol that offers no guarantees for sequence order or error detection and correction        ARP 412 is the address resolution protocol        TCP 411, the transmission control protocol, is connection-oriented, sends packets in-order, and does error checking and correction using acknowledgements, checksums, flow control, retransmit and sequencing        UDP 411 is user datagram protocol, a fast and unreliable protocol        Telnet 410 is a protocol for remotely logging into other computers on the network        NFS 409 is the network file system, a de facto file access standard created by Sun Microsystems        DNS 409 is the domain name services        FTP 408 is the file transfer protocol used to exchange files between computers        SMTP 408 is the simple mail transfer protocol which is used by email clients and servers for exchange electronic mail        
FIG. 5 illustrates how multiple networks are integrated within a computer using the TCP/IP model. In this example, we assume the computer can access three networks: a Local Area Network (LAN), an 802.11b Wireless LAN (WiFi) network, and a GPRS Wireless WAN network. The computer requires an interface card for each network, represented by the appropriate Network Interface Card (NIC) 506. Each NIC comes with a software driver 505 that converts the physical signals from the network into the transport protocol that the computer understands. The driver also enforces security that is required for that network. Each NIC is assigned an IP address by the network, which the operating system uses to route traffic using that NIC. By plugging into the appropriate stack on the operating system, the network is transparent to the user and application 501 that sits on top of the networking stack. This separation into layers makes it very convenient because neither users nor applications need to be concerned about which network is running In addition, the application developer does not have to port the software to various network transports but only has to write to the highest level provided by the underlying operating system such as HTTP or sockets. The operating system takes care of loading the various drivers of the NIC's to enable the networks. Different operating systems will have different policies for which network has precedence. Since the applications cannot discern which network is running and the networking philosophy is based on one network being available, operating systems have to decide which one is the default network (or default route). Often, the latest network that was loaded is the default route. This means that all traffic goes through that network even though the other networks are available. The other networks are still available and can be directly used by addressing it via its IP address. Some applications will want to route traffic from one network to another; such as router software. The operating system keeps track of the networks in a route table 507 and this determines the precedence of each network as well as the default route.
Some operating systems allow static configuration settings that set up simple rules or policies on how to handle multiple networks. The administrator of that computer must be very knowledgeable to set up this configuration. But since this is static, there is no way to change the default route based on specific characteristics of the application data (eg. very large files), system parameters (eg. time, battery level) or cost of using a particular network.
The assumption is that a network is always available once it is up. If it is not available, then a timeout occurs, resulting in unpredictable application behavior. With the advent of Wireless WAN's, applications need to be intelligent to handle network outages because a wireless network will not always be available. In addition, certain networks are more expensive to use than others (eg. Satellite) and should therefore be used sparingly and only when high priority messages are to be sent or received.
Where there is an important need to decide which network to use, applications today have been specially written which know exactly what types of network to use and have hard coded policies to decide when there are multiple networks available.
In addition, the network routing philosophy of “least cost routing” simply looks at the currently available networks and sends messages on the route that it deems to be the lowest cost. However, there are times when that is not desired, for instance, when a message should be sent only using a particular type of network resource or cheaper, otherwise, it should be held on the device because it is not of any particular urgency to be sent.
5. Integrated Development Environments
There are many Integrated Development Environments (IDE) available on the market for developing client and server applications. Among the dominant players are Microsoft's Visual Studio, IBM's WebSphere Application Developer and Sun's NetBeans platform. There are versions that are modified to develop mobile applications on the most common platforms such as Microsoft Windows CE, PalmOS, RIM OS, J2ME and Personal Java. These IDE's subscribe to the procedural method of programming for the device. In other words, the developer has to write a lot of code describing exactly what the application has to do. The advantage is a lot of control over the specific look-and-feel and behavior of the application. The downside is that the developer has to port to application to every different target platform. For example, the Windows CE program must be rewritten for the Palm or the RIM Blackberry. However, with the many different form factors of devices, this will result in a lot of additional development and maintenance to support multiple platforms.
There is another popular paradigm, which is the forms-based methodology for creating applications. This is useful to define database-centric applications that do queries and display the results, or for web applications where HTML is generated. Oftentimes, scripting is provided as an option to further specify behaviors. But this method does not give the low-level control that many developers want. This is important because the small screen and form-factor makes usability a paramount issue in handheld software design.
An alternative methodology is based on the declarative model, where business objects are modeled and their data is poured into graphics objects. This model is often used by packaged applications (eg. PeopleSoft) to customize the modules because the business objects and GUI (Graphics User Interface) are all well defined. The customization is exposed to the user via a set or property sheets. It is a powerful methodology because it enables the most productive development environment by generating most of the “glue” code between the business and graphical objects. However, it suffers from the same shortfall as the forms-based paradigm, which is the lack of low-level control over graphical objects.
It would be ideal to have an IDE that is based on the declarative model that a developer can use to create general loosely-coupled client-server applications. The IDE should also provide the ability to import specialized graphical objects in order to allow fine control over the behavior of the application, which is critical to usability.
KonaWare proposes this type of IDE using XML as a specification language, thereby making it open and not locking the customer into any specific environment. In addition, this can be implemented as a standalone program or as a plug-in to the popular IDEs.
6. Mobile Device Management
There is an emerging market segment for mobile device management tools because of the proliferation of mobile devices; starting first with the laptop and now with the different types of PDA's and tablet PC's that run various operating systems from Microsoft, Symbian, RIM, Palm, Linux, etc.
Until today, mobile device management largely consists of managing Microsoft Windows-based laptops. Functions such as asset management, software deployment, security management, configuration management and automatic backup/restore are some of the common features in the vendors' offerings.
Managing devices that are connected via intermittent networks, or multiple networks, presents new challenges and requirements. The management agent on the device needs to have a reliable asynchronous messaging communication with the server because the connection could drop at any time. For software deployment and backup/restore, there needs to be a provision for selecting which network to use since it might not make sense to send large updates through low-bandwidth and intermittent wireless WAN's. The management agent must be able to run diagnostic tests, reconfigure the settings should they be corrupted, and send regular reports back to the server.